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Since its introduction to the education arena in the 1960s, computers have both 
intrigued and frustrated teachers and researchers alike. The many promising 
prospects of computers and its applications did not materialise, and research into 
their effectiveness in learning has left many unanswered questions. The methods used 
in educational research of this nature in the past and present have evolved over the 
years. Quantitative studies such as meta-analyses are still widely used in the United 
States while recent large-scale research in United Kingdom used a combination of 
quantitative and qualitative methods. Findings from these research studies have 
indicated small positive effects and consequently a need for more in-depth and 
longitudinal studies into the impact of ICT on learning in the future. 

ICT, qualitative analysis, quantitative analysis, meta-analysis, learning 


INTRODUCTION 

With the introduction of computers, the precursor of our modern-day ICT, and the promising 
potentials of computer-based instruction and learning, many researchers and funding agencies 
were led to invest much of their resources to investigate the possibility of computers replacing 
teachers in key instructional roles (Roblyer, Castine and King, 1988). Moreover, the ‘Everest 
Syndrome’ (cited in Roblyer et ah, 1988, p. 5) also resulted in many believing that computers 
should be brought into the education arena simply ‘because they are there’ and the resultant 
perpetuation of the myth that students would benefit qualitatively from computers by simply 
providing them with the software and hardware. 

However, this initial enthusiasm and novelty effect began to diminish as the realisation that the 
fulfilment of the promises and beliefs was not forthcoming, became increasingly evident. 

Reynolds (2001) in his keynote presentation on ‘ICT in Education: The Euture Research and 
Policy Agenda’ lamented that 

.... we are trapped in a cycle of classic innovation failure - a low quality 
implementation of a not very powerful new technology of practice produces poor or no 
improvement in outcomes, which in turn produces low commitment to the innovation 
and a reluctance to further implement more advanced stages of the innovation... that 
are more likely to generate the improvement in outcomes that would produce the 
commitment to ICT utilisation. (Reynolds, 2001, p.2) 

SCOPE OF REVIEW 

This review seeks to examine and understand the methodology used by researchers to study the 
impact of ICT on learning. The findings from these research studies will help to evaluate its 
effectiveness on students’ learning outcomes and implications for education and further research. 
Most of the studies reviewed are limited to the United States and the United Kingdom, where 
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research in this field has been more consistent and well documented. Two periods of research 
have been suggested in this review. 

(a) Research findings and their implications from 1960s to 1980s; 

(b) Research findings and their implications froml990s to 2000s, and future research. 

METHODS OF ANALYSIS 

The Qualitative Approach 

In-depth case studies of small groups of learners are usually the norm in qualitative methods of 
research. Detailed records of ICT-related activities, as well as the learning taking place, are 
essential as they are necessary for the identification of relationships between them. However, 
because of the group size being investigated, it is often difficult to generalise any findings from 
such studies as they are not representative of the whole school population or community. 

The Quantitative Approach 

The quantitative approach often involves an experimental (or treatment) and a control group. The 
experimental group is directly involved in the ICT-related learning activities while the control 
group learns using the traditional method. Both groups are tested before and after the experiment 
and sometimes, a delayed test may be given to determine the retention rate of the learning. One of 
the limitations of the quantitative approach is that other factors, such as a novelty effect involving 
increased enthusiasm of teachers and students, may be unconsciously introduced to confound the 
results of the experiment. 

The Quantitative-Qualitative Approach 

In combining both qualitative and quantitative methods, a greater degree of accuracy and validity 
in the results of studies is obtained, thus strengthening the findings and implications put forward 
by the researcher. Two methods of this combined study have been advocated. The first method 
involves the conducting of a large-scale quantitative study, followed by case studies of in-depth 
investigation (Becta, 2001; Cox, 1993). The second method is a well-established approach known 
as meta-analysis. In this method, a large number of published case studies of similar 
characteristics are collected and comparative analysis made to identify relationships between these 
variables. Since its inception, researchers have consistently used this method to investigate and 
evaluate data in their research. This method is described in greater detail in the next section. 

THE META-ANALYSIS METHOD 

The meta-analysis technique was pioneered by Glass (1977) and later adopted by many reviewers 
(Cohen, 1981; Kulik, Bangert and Williams, 1983; Roblyer, 1988) in their research. Meta-analysts 
(Kulik et ah, 1983) normally used a quantitative approach to their studies incorporating three main 
tasks: 

(a) objective procedures to locate studies; 

(b) quantitative or quasi-quantitative techniques to describe study features and outcomes; and 

(c) statistical methods to summarise overall findings and to explore relationship between study 
features and outcomes. 

The procedures involved, as used by Kulik and his associates (Kulik, Kulik and Cohen, 1980), are 
briefly described. 
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1. A large number of studies that examined the effeets of computer-based instructions are 
collected from different databases. 

2. Guidelines are used to sieve through the studies collected and those that fail to meet the 
criteria are removed. Each study is counted once even when it is presented in several papers. 

3. Variables and categories for describing features of the study are developed; Experimental and 
control groups are taught during the same period and objective examinations are used as the 
criterion of student achievement. Attitudes toward computers, subject matter and instruction 
are based on self-report responses to questionnaire items or scales. 

4. To quantify outcomes in each area of study, the Effect Size (ES) is used. The effect size is 

defined as the difference between the means of two groups divided by the standard deviation 
of the control group. — — 

Effect Size = 

SDc 

where X is the mean of the experimental group, C the mean of the control group and SDc 
is the standard deviation of the control group. 

An extract of the results obtained in the study by Kulik and his colleagues (Kulik et al., 1980, p. 

23) is shown in Table 1. 

Table 1. Means and standard errors of achievement effect sizes for different categories of 


studies 


Coding Categories 

Number of 
Studies 

Effect Size 

Mean 

Standard Error 

Managing 

11 

0.33 

0.12 

Tutoring 

11 

0.36 

0.18 

Simulation 

5 

0.49 

0.33 

Programming 

8 

0.20 

0.08 

Drill and Practice 

11 

0.27 

0.11 


In order to include the impact of the given effect’s size, Cohen (1977, p. 184-185) gives the 
following guidelines for effect size: 

ES of 0.2 or less = small effect 

ES of 0.5 - 0.8 = medium effect 

ES of 0.8 or more = large effect 

5. Einally, the direction and significance of differences in instructional outcomes between the 
control and experimental groups were also examined and scored using a four-point scale (see 
Table 2). 


Table 2. The four-point scale 


Scale 

Difference Outcome 

Statistically 

Significant 

1 

Favoured Conventional Teaching 

Yes 

2 

Favoured Conventional Teaching 

No 

3 

Favoured Computer-Based Instruction 

No 

4 

Favoured Computer-Based Instruction 

Yes 


6. As correlation between the effect size and the four-point scores is usually high, regression 
equations are used for ‘plugging’ effect-size measures for cases with missing data. 
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RESEARCH (1960s - 1980s) 

Many studies were eonducted in the past to evaluate the effectiveness of computers in the learning 
environment. The earliest took place in the 1960s and 1970s when researchers introduced pupils 
to educational software in a university environment (Cox, 2003). In those studies, learners did not 
use ICT in a normal classroom setting or within their subject curriculum, but were using software 
specifically designed to address specific conceptual difficulties in subjects such as science or 
mathematics. Other studies in the 1970s measured the impact of learning through traditional pre- 
and post-tests using experimental and control groups. Their performance was usually assessed by 
the conventional end-of-year examinations. 

Box Score Method 

In the early years of research, there was no reliable procedure for the reviewing of studies 
(Roblyer et ah, 1988). As such, the so-called box score method was the only available method and 
it involved the following steps: 

• collect all the experimental and evaluation studies; 

• examine each study to determine the significant difference between the experimental and 
control studies; 

• count the number of studies which did and did not find such differences. 

In the area of instructional computing, the box score method of summarising results was 
especially problematic where differences in treatment effects were rarely large enough to reject the 
null hypothesis and most studies tended to show non-significant differences. Edwards et al. (1975) 
completed a review on computer-assisted instruction using the true box score. They located some 
30 studies and coded them for type of CAI, subject area, grade level, supplemental or replacement 
use, and results. Their findings indicated that CAI was more effective at elementary levels, and for 
supplemental use. It was also equally effective individual tutoring and programmed instruction, 
and found to reduce learning time. 

Major Studies (1960s - 1980s) 

Unlike today, computers were not as widely available in the early days of microcomputers. In 
spite of this limitation, a few major researches were advanced especially in the area of technology- 
intensive programs. Some of these studies are highlighted below. 

The Los Angeles Unified School District Study 

The Los Angeles Unified School District used the Computer Curriculum Corporation (CCC) 
program for its study in six of its schools in 1976. The CCC curricula used included drill and 
practice activities in mathematics (Grades 1-6), reading (Grades 3-6) and language arts (Grades 3- 
6). Pre- and post-tests were used to measure achievements. Of the six schools, four received 
computer-assisted instruction while two acted as control. Within the CAI schools, alternate groups 
of students either received CAI or acted as controls (did not receive CAI). 

The schools under study had to use the system for drill 10 minutes each school day. Over 2000 
students were involved in the study for a three-year period. The final results showed effect sizes of 
0.45 for mathematics computational skills, 0.10 for mathematics concepts and applications and 
0.15 for reading and language. 
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Project IMPA C 

Four different set-ups were used in this project known as IMP AC (Instructional Microcomputer 
Project for Arkansas Classrooms) (McDermott, 1985). The set-up was to allow the use of a 
computer by each student for a certain period of time each day or week. The arrangements were as 
follows. 

• Three types of software were loaded into four microcomputers. Students used the four 
microcomputers from three to ten days in a ten-day cycle for about 12-20 minutes. 

• Eight microcomputers in a network were placed in classrooms where teachers worked with 
two or three mathematics groups. Two comprehensive mathematics packages were used. The 
usage by the students was similar to the first group. 

• Twenty-four microcomputers were placed in a classroom. Students were brought to the 
classroom and used the computers for 15-25 minutes each day for eight days out of ten. 
Students used the same software as that used by the second group. 

• Eight computers were networked and a locally-developed software package was used to 
monitor student progress. Computer-aided instruction was used to supplement the traditional 
mathematics program. The same time schedule was used as that used by the second group. 

Effect sizes of 0.02 to 0.62 were reported for the study, with the most favourable results coming 
from the last group where traditional instruction was supplemented by computer-aided instruction. 

Minnesota Technology Demonstration Program 

This was a major two-year study of microcomputer use with Grade 4 to 6 students in Minnesota 
schools. Over 20 per cent of Minnesota school districts were involved and both microcomputer 
and video-based technologies were used. The computer to student ratio was very favourable and 
computer software was selected and used extensively and systematically based on skill needs 
(Morehouse, 1987). 

The final results indicated that the average effect sizes achieved in mathematics, reading and 
language were -0.09 to -0.31, a rather disappointing finding after much initial enthusiasm and 
optimism from the participating teachers. 

Meta-analysis studies by Kulik 

In 1983, Kulik and his colleagues (Kulik et ah, 1983) conducted a meta-analysis of 51 
independent evaluations of computer-based teaching in Grades 6-12 and found that computer- 
based teaching raised students’ scores on final examinations by approximately 0.32 standard 
deviations, or from the 50**^ to the 63"^'* percentile. Smaller, positive effects on scores were also 
evident on follow-up examinations several months after the instructions. In addition, they also 
discovered that students developed positive attitudes toward the courses that they were taking and 
learning time was substantially reduced with computer-based instruction. 

In an updated analysis of the effectiveness of computer-based instruction, Kulik and Klulik (1991) 
conducted findings from 254 evaluation studies based on studies prior to 1990. The outcomes 
measured were: 

• student learning; 

• performance on a follow-up or retention examination; 

• attitude toward computers; 

• attitude towards school subjects; 

• course completion; and 
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• amount of time needed for instruction. 

A total of 248 of the 254 studies reported results from CBI and control groups on examinations 
given at the end of instruction. In 202 of these studies, the students in the CBI had the higher 
examination average while 46 of them showed that students in the conventionally taught class had 
the higher average. The difference in the examination performance of the CBI and control 
students was reported to be significant in 100 studies. 

The average effect size was 0.30 with a standard error of 0.029, again confirming his earlier 
findings. This means that the performance of the CBI students was 0.30 standard deviations 
higher than the performance of the control students. In other words, the average student from the 
CBI class would outperform 62 per cent of the students from the conventional classes. 

Past Reviews, Findings and Implications 

In their review, Roblyer et al. (1988) made comparison studies before and after 1980 and 
presented their findings. In the pre-1980 studies, nearly all the estimated 200 studies indicated 
positive evidence that computer-based treatments offered some benefits over other methods, 
although a clarification was that there were few clear disagreements among the reviews. A 
summary of the findings indicated: 

• reduction in learning time; 

• limited improvement in motivation toward learning; 

• computer-based treatments were generally effective in mathematics and reading/language; 

• computer-aided learning (CAI) was more effective as a supplement at lower grade level; 

• slow learners and under-achievers seemed to gain from computer-based methods than more 
able students. 

• computer-based methods are generally more effective at lower grade levels. Effectiveness of 
computer-managed instruction (CMI) seems to increase at higher grade levels while CAI 
effects seem to decrease at higher levels. 

For the post-1980 review (Roblyer et ah, 1988), positive effects were for achievements in every 
analysis of the 85 studies except for ESL, problem-solving CAI, achievement in females and 
attitudes toward computers as instructional media. The review concentrated on five main areas as 
shown below. 

• Attitudes - Only three studies with available data were used in the review and all showed a 
consistent positive trend towards computers. 

• Content Area - Computer application was most effective for Science, followed by 
Mathematics, Cognitive skills and Reading/Eanguage. 

• Application Type - Analyses were made for Reading and Mathematics due to sufficient 
numbers of data and both were found to be equally effective. 

• Grade Level - Of the three levels, effectiveness of CAI was highest for college/adult level 
(ES = 0.57), while that of the elementary and secondary levels recorded effect sizes of 0.32 and 
0.19 respectively. 

• Types of Students - There was indication of greater effectiveness of computer application with 
lower-achieving students than with regular, on-grade students though the difference in effects 
was not significant. 
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Recognising the need and urgency for research in this field as its priority seemed to be losing 
grounds, Roblyer et al. (1988) called for a change in the perception about the value of behavioural 
research in several ways. 

• Educational organisations must change their perspectives on research, accept it as a 
requirement for development and growth, and give it a funding priority along with hardware 
and software. 

• Funding agencies must accept the need for more and better research in instructional computing 
methods, provide the funds to support it, and solicit ongoing research projects to answer key 
questions. 

• Practitioners must begin to rely on the results of research to indicate the validity of their beliefs 
and hypotheses, and insist that their organisations provide the data that are needed. 

Further research on different areas was also encouraged in the light of the findings. These areas of 
research are listed as shown: 

• application in various skill and content areas; 

• computer applications in ESF; 

• word processing use; 

• creativity and problem-solving with Fogo (a programming language) and CAI; 

• effects of computer use on attitudes and drop-out rate; 

• effects of computer use on achievement of males vs females. 

RESEARCH (1990s - 2000s) 

Much educational research on ICT has been conducted over the past ten years, with more large- 
scale studies evidently from the United States and the United Kingdom though there are reports of 
research in different parts of Europe. Fiterature reviews in this field are important not only to 
educators but to policy makers who are usually reluctant to fund large-scale longitudinal studies. 
Yelland (2001) reported the need for such funding in Australia to support a variety of research 
studies which should include a mixed-method research design (Yelland, 2001, p. 36). 

Such research would recognise positive effects and identify any negative influences. In 
this way we could determine how best to promote effective learning so that outcomes 
are improved. 


American Studies 

Though American educational research has always been in the forefront, its educational system 
has been heavily criticised by the American public over the past several decades (Christmann, 
2003). Contributing significantly to the criticism of the public schools is the dismal placement of 
the nation’s mathematics and science students within the global hierarchy, for example The Third 
International Mathematics and Science Study (TIMSS). In response to the public outcry and 
criticisms, the schools are incorporating CAI into their curricula in efforts to enhance student 
achievement. 

In order to answer the question, “What differences exist between the academic achievement levels 
of elementary students who were exposed to computer-assisted instruction, and those who were 
not exposed to this instruction during consecutive years?”, Christmann (2003) conducted a meta- 
analysis of 1800 studies for students (in grades K through 6) adopting the following criteria. 

1 . They were conducted in an educational setting; 
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2. They included quantitative results in which academic achievement was the dependent variable 
and microcomputer-provided CAI was the treatment. 

3. They had experimental, quasi-experimental or correlational research designs. 

The sample sizes had a combined minimum of 20 students in the experimental and control groups. 

Only 39 out of the 1800 studies qualified for the final inclusion in the research. A total of 8274 
students were involved, with sample sizes ranging from 20 to 930 and a mean sample size of 122 
students. The result of the test recorded a mean effect size of 0.342, confirming that higher scores 
were attained by students receiving CAI, though this effect is considered small (Cohen, 1977). 

In another meta-analysis study on the effectiveness of CAI on student achievement in secondary 
and college science education was compared to traditional instruction, Byraktar (2001/2002) 
reviewed 42 studies between 1970 to 1999. In calculating the effect size, she used the formula 
devised by Hunter and Schmidt (Hunter, 1990) where a pooled standard deviation is used instead 
of the standard deviation of the control group as developed by Glass (1977). 

In the final analysis, only one of the 42 studies showed no difference between CAI and the 
traditional instruction group (ES = 0). The range of the ES was from -0.69 to 1.295. The mean 
effect size was 0.273 which can be interpreted as an average student exposed to CAI exceeding 
the academic achievement of 62 per cent of the students in the traditional classroom. 

Waxman and his associates (Waxman, 2003) from the University of Houston synthesised recent 
research on the effects of teaching and learning with technology on student outcomes. This 
quantitative study sought to investigate the following questions; 

• How extensive is the empirical evidence on the relationship between teaching and learning 
with technology and student outcomes? 

• What is the magnitude and direction of the relationship between teaching and learning with 
technology and student outcomes? 

• Are there certain social contexts or student characteristics that affect the relationship? 

• Are there particular methodological characteristics that affect the relationship? 

• Are there specific characteristics of the technology that affect its relationship with student 
outcomes? 

• Are there specific characteristics of instructional features that affect technology’s relationship 
with student outcomes? 

The synthesis included quantitative, experimental and quasi-experimental research and evaluation 
studies during a six -year period (1997-2003). The study also focused on studies that have teaching 
and learning with technology in K-12 classroom contexts where students and their teachers 
interact primarily face-to-face, compare a technology group to a non-technology comparison 
group and have reported statistical data that allowed the calculation of effect sizes. Using these 
criteria, the study was trimmed down from an initial size of 200 to 42. The final study contained a 
combined sample of about 7,000 students with a mean sample number of 184. The mean of the 
study-weighted effect sizes across all outcomes was 0.410 (p<0.01), with a confidence interval of 
0.175 to 0.644. This result indicated that teaching and learning with technology had a small, 
positive, significant effect on student outcomes when compared to traditional instruction. 


British Studies 

In Britain, the National Council for Educational Technology (NCET) and its successor 
organisation, British Educational Communications and Technology Agency (Becta) have taken 
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large and bold steps in sueh educational research. During the mid-1990s, the United Kingdom 
government invested in a large-scale evaluation of integrated systems (Cox et ah, 2003). The first 
report released by NCET was positive and found that children had made learning gains in 
mathematics, although not in reading. A second report identified the transferability of learning 
gains. The final report made an important reservation that the gains did not appear to be 
automatically transferable. The researchers also reported that the software used was mainly seen 
by pupils and teachers as being successful in teaching core mathematical and English skills but 
not all such successes were measurable through the subsequent tests or examinations (Wood, 
1999). Effects on motivation and behaviour were marked but there was no evidence that they 
transferred to other context. 

Two more large-scale projects by Becta, ImpacT and ImpaCT2 were initiated to examine the 
impact on ICT and attainment by students. 

ImpacT Project 

This United Kingdom project developed a range of assessment methods based on those used by 
previous large-scale projects, as well as new ones which were specifically designed to measure 
attainment in conceptual understanding and intellectual processes (Watson, 1993). These 
included: 

• different subject- and topic-based tests, conducted over two years, testing pupils’ subject 
knowledge at the beginning and end of the period; 

• a series of linked case studies to investigate the effects of teachers’ pedagogies on pupils’ use 
of ICT and the consequent learning outcomes; and 

• a study of the uptake and use of ICT by all the teachers and pupils in the study, which involved 
new data-collection instruments, including pupils’ record sheets (collecting data on the types of 
ICT use and where this occurred). 

In this study, there was evidence of a positive contribution to attainment in English, with a 
statistically significant effect of using word processing for pupils aged 8-10 years but only a 
partial non-significant effect for pupils aged 12-14 years. The pupils’ English was assessed 
through various essay-writing tasks, which were graded by two independent English teachers. The 
quality of the essays was also assessed through measuring the rates of cohesion and coherence in 
the pupils’ texts and the errors in spelling. The main finding from the study of primary pupils was 
that the frequency of use of ICT in their English lessons affected their achievements in English. 
There was a positive contribution from the use of word processing in the ‘high IT’ primary 
classes. When pupils composed directly with word-processing facilities, they were more prone to 
summarise and remove redundant information. At the secondary school level, the results were less 
conclusive, partly because of poor returns on the English essays and because of the limited use of 
ICT in English lessons. 

In mathematics, pupils aged 8-10 years and 14-16 years in classes which were using Logo (a 
programming language) and subject-based mathematics software achieved statistically higher 
scores in tests than those pupils who were taught similar concepts through traditional methods. 
The results provided significant evidence of a positive impact of ICT on pupils’ learning in 
mathematics where ICT were being integrated into the mathematics curriculum. The project’s 
mini-studies provided additional evidence of positive effects of ICT on attainment in 
mathematical reasoning using Logo and boolean logic skills using databases. 



644 


The impact of ICT on learning: A review of research 


ImpaCTl Project 

ImpaCT2 was one of the most eomprehensive investigations into the impaet of ICT on attainment 
eondueted in the United Kingdom (Harrison, 2001). This large-scale evaluation study, was funded 
by the Department of Education and Skills and managed by Becta. The study extended over three 
years (1999-2002) and its purpose was to make an independent evaluation of the impact of ICT on 
children’s achievement in a representative sample of schools in England. 

ImpaCT2 was a longitudinal study involving 60 schools in England and its aims were to: 

• identify the impact of networked technologies on the school and out-of-school environment; 

• determine whether or not this impact affects the educational attainment of pupils aged 8-16 
years; 

• provide information that would assist in the formation of national, local and school policies on 
the deployment of ICT; 

• devise methods of assessing pupils’ attainment; and 

• devise a framework for measuring the ICT environment. 

This study combined both traditional and new research methods on a very large sample of more 
than 2000 pupils. Two preliminary studies were carried out in 1999 to determine the appropriate 
methods for measuring the impact of ICT. 

The sample studied was 2,179 pupils in 60 schools, of which 30 were primary, 25 secondary and 
five were special schools. The selected schools represented pupils from different socio-economic 
groups in the urban, suburban and rural areas of England. The schools were divided into two 
groups: one group having a high quality of ICT provision and the second group having an average 
ICT provision. In each school, teachers selected 25 pupils from each key stage (KS) as a 
representative sample of the children in their schools in terms of ability, gender, ethnicity and 
socio-economic status. Data were collected on 25 pupils from each key stage. 

• KS2 (Year 5 in 1999/2000); 

• KS3 (Years in 1999/2000); 

• KS4 (Year 10 in 1999/2000). 

Three strands of analysis were incorporated into the study: 

• Strand 1 : analysis and interpretation of national test data in relation to school rating for ICT; 

• Strand 2: development and use of innovative research methods to reveal how pupils use ICT 
out of school; and 

• Strand 3: independent Triangulated Case Studies. 

In Strand 1, historical test data for individual pupils was extrapolated to indicate predicted 
performance. Actual performance was then compared with the predicted test results. The purpose 
of this analysis was to measure whether effective use of ICT by a school could enhance pupils’ 
attainment. In Strand 2, teacher researchers, pupil researchers and link (professional) researchers 
gathered data on ICT activities outside the school. In addition, a pupil questionnaire and a concept 
mapping task were used to gather information and data from the pupils involved in the project. In 
Strand 3, an independent team of researchers from the University of Eeicester carried out 15 case 
studies of a representative sample of primary, secondary and special schools. The techniques 
involved the use of video diaries and electronic journals of ICT practice created by pupils and 
teachers. 
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The study reported mixed results for the effeets of ICT on pupils’ attainment in English. At the 
primary level, there was a statistically significant impact of ICT on the KS2 English tests but not 
at KS3 or KS4 (Harrison, 2002). Case studies also showed the predominant use of ICT in English 
was for word processing (Comber, 2002). 

Evidence from the study showed that ICT had a positive relationship to pupils’ learning of 
mathematical skills and the results varied according to the amount and type of use of ICT in the 
mathematics curriculum. High users of ICT at KS3 outperformed, on average, low users of ICT in 
mathematics, but differences at KS4 were slight. However, this aspect of the research was 
designed to establish whether correlation existed between ICT use and attainment and not to 
establish causal relationships. 

The relationship between ICT use and attainment in KS2 was least marked in science, failing to 
reach statistical significance. However, findings at both KS3 and 4 indicated statistically 
significant positive associations between the level of ICT use and pupils’ attainment and showed 
an enhanced performance. 


Findings and Implications 

Meta-analysis studies of America schools generally showed a positive trend in the achievement of 
students when ICT is employed (eg Christmann, 2003, Bayraktar, 2001/2002, Waxman, 2003). In 
her studies, Bayraktar (2001/2002) found that computers were more effective when used in 
simulation or tutorial modes and CAI was more effective when computers were used individually 
by students. She suggested that tutorial and simulation CAI programs could be used in science 
classrooms to enhance student learning and classrooms should have sufficient computers for 
students to work individually. Eurthermore, the studies showed that CAI was more effective when 
used as a supplement to traditional instruction rather than as a substitute. Therefore, the use of 
ICT in conjunction with other teaching strategies could be more beneficial for student learning. 
Another important finding revealed that teacher-developed programs were more effective than 
commercial software programs. Consequently, more attention should be devoted to specific 
educational objectives and curriculum goals when designing software for higher levels of 
effectiveness. 

In ImpaCT2, the findings (Harrison, 2002) indicated that a diversity of pupils were learning with 
ICT. There was evidence that some pupils were gaining benefit at all the key stages investigated. 
Evidence from the qualitative strands of the research strongly suggested that impact on the 
curriculum was greatest when pupils’ use of ICT was fully integrated across the curriculum as a 
whole through both classroom and home-based activities. Although the findings from the study 
covered many areas, only aspects relating to the learning and ICT are listed as shown. 

• Positive effects on school achievement for higher usage levels of ICT (based on pupil estimates 
of ICT activity) were found both at the level of the individual pupil and at the level of the 
school, although these were not large. 

• The relationship between levels of ICT usage and effectiveness was not consistent across all 
key stages and subject areas at the school level. 

• In none of the 13 comparisons was greater use of ICT associated with poorer outcomes, 
whether in terms of relative gains or in terms of examination results 

• Pupils are engaging in innovative uses of technology often outside the school context and are 
acquiring a complex range of skills and literacies in networked ICT, including a range of on- 
line social and communication skills. 
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• Many pupils have developed a eomplex understanding of the role of eomputers in the world 
today ineluding a wide range of equipment and loeations in whieh sueh teehnologies are used. 

In view of these findings, some issues are being eonsidered in the sehool eontext (Harrison, 2002). 

• The aceeptanee that networked teehnologies in sehools are inevitable and benefieial is almost 
universal among teaehers. However, many of them are as yet unsure as to the impaet of ICT on 
attainment, although they acknowledge other benefits such as increased motivation and 
improved behaviour. ICT is perceived to be particularly beneficial for pupils with special 
educational needs. 

• Schools should concentrate their attention on using ICT in the teaching of curriculum subjects 
as an aid to improve attainment. 

• Strategies for effective use of ICT resources, particularly searching on the Internet a'nd use of 
ICT to support homework are still in development. 

FUTURE RESEARCH 

There is clear evidence from the present findings that ICT has a positive, although small effect on 
the learning of students. Most researchers appear optimistic about the roles that ICT will play in 
the school environment in the future though some have their reservations. In his concluding 
remarks, Christmann (2003) wrote; 

.... more research is needed to test the lofty expectations that many have CAI as the 
basis for educational achievement. Otherwise, CAI may become another 
misunderstood, overbought, under-used, and, eventually, a large discarded tool. 

Indeed, with their updated knowledge and acquired experience on ICT, researchers are even 
keener to investigate thoroughly its effectiveness on learning, acknowledging that its continuing 
existence in schools is inevitable. Increasingly, there appears to be an underlying call for large- 
scale and longitudinal study of ICT and its impact on learning worldwide. 

Based on evidence gathered from published research studies, especially in the context of ImpaCT2 
findings, Cox (2003) in her report to the Department for Education and Skills, Becta, 
recommended five key areas of priorities for future research. 

1 . A need for more long-scale studies in order to: 

• measure attainment which is sustained over a long period (at least two to three years); 

• find out what specific uses of ICT have on the learning of concepts and skills in specific 
topics and subjects; 

• monitor and assess the whole learning process; 

• compare the effects of different uses of ICT on the learning of the same subject; 

• measure the effects of the use of ICT on the curriculum, and consequently on the learning 
of the pupils; and 

• identify appropriate methods for measuring the effects of specific use of ICT to take 
account of new ways of learning and new knowledge. 

2. Research needs to be conducted to measure how informal learning experiences contribute to 
the whole learning process and thereby affect learners’ achievements. 

3. New methods of measuring attainment need to be developed. 

4. More research needs to be conducted into the effects of specific uses of ICT on pupils’ 
approaches to learning generally, on their meta-cognitive skills and on their long-term 
learning strategies. 
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5. A more extensive review of the literature would provide more substantial evidenee of the 
effects of specific uses of ICT on pupils’ learning. 

Australian Research Perspective 

Over recent years, there were various Australian initiatives and research in the area of ICT, 
generally qualitative in nature. In 1999, a commissioned study (Downes, 2001) was conducted on 
the educational use of the internet with children eight years and under. The data for the study were 
obtained through literature review, a one-day workshop, telephone discussions, two 
observation/discussion sessions with children, a focus group of parents and telephone and/or 
email discussions. Findings indicated the need for sound pedagogical framework in the use of 
digital resources so as to ensure their effective use with young children. 

In addition, Netdays Australia 2000 (Carr, 2002) was a case study involving 60 teachers and 342 
schools around Australia and 15 classes in overseas countries. Its aim was to examine the 
teacher’s role in an online curriculum project. The data provided by the teachers were gathered 
through three surveys and a detailed report from 14 teachers who successfully implemented the 
online project. The main learning outcome for students was to recognise, promote and celebrate 
cultural diversity. However, there were no guidelines in assessing students in the online projects 
and teachers had to develop their own criteria for the evaluation of their students’ work. Six 
pedagogical traits were identified from the study. 

• Teachers saw themselves as learners with their students and managers of their students’ 
learning. 

• The technology matched the needs of the students and the project. 

• The teachers structured the activities so that support was provided in different areas as the 
project grew. 

• The teachers were totally committed to the project. 

• The teachers had in place common management strategies. 

• The students felt ownership of the project. 

In a joint investigation by the Deakin Centre for Education and Change, The Institute of Koorie 
Education and the Institute of Disability Studies at Deakin University, Blackmore and her 
colleagues (Blackmore et ah, 2003) reported that there was a lack of evaluation of effective 
practice using ICT in schools. Although ‘no research was undertaken in schools’ (Blackmore et 
ah, 2003, p. i), their report was developed through: 

• workshops with a reference group that developed the conceptual framework and identified key 
words, 

• focused discussions between researchers in specific areas, 

• contact with teacher networks, 

• contact with ICT policy personnel in state educational systems, 

• reference list of 1000 items, and 

• 200 websites. 

The report also suggested that “there is still little evidence about how it (ICT) impacts on 
cognitive learning outcomes” (Blackmore et ah, p. 209/210). 
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Paris’ (2004) latest work involved 52 Year 10 students from South Australia. The study was to 
examine students’ attitudes towards online web assisted learning (OWAL). Using data collected 
through questionnaires, one of the findings was that students showed a strong positive tendency 
towards OWAL compared to paper assisted learning, that is the use of text. 

However, these Australian studies do not seem to address the basic question, “How can we 
measure the effectiveness of ICT on students’ learning outcome?” Consequently, there would 
appear to be a major need for an approach to the studies of ICT in Australia that emphasise both 
the quantitative and qualitative aspects. The findings would then lead to more effective teaching 
and learning for students, which is the chief aim of education. 

Yelland (2001) stressed the need for an Australian research on the impact of ICT on learning, 
especially with regard to literacy and numeracy outcomes and the concept of multi-literacies that 
incorporate the use of ICT. This need arises from the fact that the majority of influential studies 
relating to ICT originate from the United States and more recently the United Kingdom. She felt 
that a quality Australian-based research study would inform educational practice and improve 
outcomes for all students in Australian schools. Furthermore, there must be a search for ways to 
improve learning and educational outcomes that include the use of appropriate technologies as an 
integral part of learning, not as an add-on to existing practices that were in place before ICT was 
invented. In addition, a longitudinal study to complement research in literacy and numeracy 
research would be beneficial. This study would document the effective use and integration of ICT 
and link them to a variety of outcome measures. 

Further Research 

Recently, there has been an increased interest in neural network and learning. The brain, which is 
the centre of learning, memory and recall, plays an important part in the whole learning process. It 
is therefore important that further research should also be focussed on these areas: 

(a) interdisciplinary research on how the brain learns and how ICT can be used effectively in the 
learning process; 

(b) the use of ICT to stimulate the memory process, leading to effective learning; 

(c) innovative computer softwares or programs and other ICT tools that help students to think 
creatively and critically. 

It is interesting to note that many students are attracted to simulation and video games, and they 
tend to spend hours playing them in arcades and at home. What are the features of these games 
that make them so irresistible to students? In fact, the game developers are way ahead in terms of 
the technology used compared to that which is available in schools. It is no wonder that students 
find the software used in schools boring and uninteresting! Research into this area may include: 

(a) a study on the features of video games that make them attractive to the players and how some 
of these features may be incorporated into educational softwares; 

(b) an extended study on the effectiveness of these new educational programs on learning; 

Many research studies, especially meta-analyses, have made findings that showed that ICT has 
small effects on learning outcomes and sometimes negative effects have been found. Qualitative 
research on how to improve the effective use of ICT through innovative methods, possibly 
incorporating a variety of ICT tools should be investigated as pilot studies. Such studies should be 
ongoing such that feedback can be obtained and methods modified to refine the teaching and 
learning process. 
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CONCLUSIONS 

Using the analogy of an ecosystem the school can be considered as a place where “dynamic 
interactions of species adapt to one another within the system” (Zhao, 2003). Zhao further 
emphasised the fact that any invading species may need to adapt to the ecosystem it enters but it 
can change the ecosystem and its native species. 

The introduction of ICT in schools can be likened to that of the invading species to the ecosystem. 
There will be a time of adjustment and adaptation by the principals, teachers and students as each 
seeks to find its place in the new learning environment and interacts with the new technology. In 
fact, the ICT development in schools generally passes through four phases, namely the emerging, 
applying, and transforming phases (UNESCO, 2002). The emerging phase is characterised by the 
purchase of computer equipment and software with teachers and administrators exploring the use 
of ICT in the school. In the applying phase, ICT is used to replace existing tasks. In the infusing 
phase, schools would have acquired a range of ICT and teachers begin to explore new ways of 
using ICT for their personal and professional practice. Finally, the last phase is realised when ICT 
becomes an integral part of the school system. Thus, the impact of ICT will be felt as it permeates 
throughout the whole school system, changing the methodology of teaching, the physical setting 
and the learning process. 

Therefore it is pertinent that ICT contributes positively to the learning in schools and for it to be 
effective, it requires the conscious effort of all the species in the school ecosystem, that is the 
principal, teachers, parents and students to make it work. 
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